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SUMMARY 


The  objective  of  the  Front  End  Analysis  of  Advanced  Combat  Feeding 
Concepts  was  to  develop  a  methodology  capable  of  increasing  the  degree  and 
directness  of  influence  of  the  'user  community'  (military  users  of 
rations)  on  the  development  of  the  Natick  RD&E  technology  base  program. 

An  analytical  approach,  consisting  of  the  development  of  a  quantitative 
model  and  the  determination  of  forms  of  compatible  data,  was  selected. 
Since  the  expense  of  conducting  projects,  in  relation  to  their  anticipated 
benefits,  are  fundamental  concerns  in  establishing  relative  value,  a 
cost/benefit  approach  was  taken.  This  approach  is  augmented  by 
identifying  and  applying  quantitative  estimates  of  the  value  users  place 
on  certain  benefits  of  rations.  Such  figures  of  relative  merit,  or 
priority  values,  are  then  incorporated  into  the  basic  cost/benefit  model, 
to  reflect  such  user-priorities.  Finally,  to  allow  for  differing 
priorities  within  the  Department  of  Defense  (DOD)  community,  the  K-Means 
clustering  algorithm  is  employed  to  identify  these  different  priorities, 
the  magnitude  of  the  sample  which  each  set  of  priorities  reflects,  and 
includes  these  factors  in  the  finalized  model. 

A  prioritization  exercise  was  conducted,  as  a  means  of  testing  the 
methodology's  utility  under  real-world  conditions.  Input  from  the  user 
community  consisted  of  approximately  450  completed  surveys  from 
organizations  representing  each  military  service  within  DOD.  Using 
existing  projects  as  'surrogate'  research  proposals,  technological 
evaluations  were  conducted  by  both  Natick  scientists  as  well  as  by 
technological  experts  from  the  civilian  community.  This  input  was  applied 
to  the  model  and  the  results  analyzed,  forming  the  basis  for  several 
conclusions.  When  used  to  evaluate  both  Military  Service  Requirements,  as 
well  as  technology  base  proposals,  the  input  from  the  user  community 
almost  always  resulted  in  the  former  type  of  proposals  being  ranked 
higher.  This  suggests  that  the  model  is  better  suited  to  evaluating  a 
single  type  of  proposal  (e.g.  only  technology  base  proposals),  rather  than 
an  entire  candidate  food  program.  Other  factors  which  must  be  considered 
when  evaluating  the  model's  output  are  a  proposal's  emphasis  on  areas 
which  may  not  be  viewed  as  benefits  by  end-users  (e.g.  shelf  life  is  more 
a  prerequisite  than  a  benefit),  as  well  as  potential  lack  of  uniformity 
regarding  the  level  of  difficulty  of  each  proposal's  objective.  However, 
the  proposed  methodology  can  produce  valuable  input  regarding  the 
prioritization  of  a  set  of  proposals. 
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FRONT  END  ANALYSIS  OF  ADVANCED  COMBAT  FEEDING  CONCEPTS 


I .  Introduction 

1 .  Background 

The  Natick  Research,  Development  and  Engineering  Center's  DOD  Food 
Program  consists  of  several  generic  types  of  research.  Military  Service 
Requirements  (MSR's)  and  Joint  Service  Requirements  (JSR's),  are  usually 
oriented  toward  near-term  military  requirements,  while  the  objectives  of 
'Technology  Base'  research  have  a  more  futuristic  time  orientation. 

The  development  of  a  technology  base  program  commences  with 
solicitations  of  proposed  research  from  the  Natick  scientific  community. 

A  technology  base  committee  then  analyzes  the  set  of  proposals  based  on 
such  criteria  as  military  relevance,  cost,  etc.  The  proposals  recommended 
for  funding  by  the  committee  are  then  considered  in  conjunction  with 
MSR's/JSR's  by  the  DOD  Food  &  Nutrition  Research  &  Engineering  Board,  who 
determine  a  finalized  'program'  of  proposals  which  have  survived  the 
transition  into  funded  projects. 

2 .  Objective 

The  objective  of  this  Front  End  Analysis  (FEA)  was  to  develop  a 
methodology  to  facilitate  greater  involvement  by  the  'user  community' 
(military  users  of  rations)  in  the  development  of  the  technology  base 
program.  This  methodology  was  not  intended  to  replace  the  current  system, 
but  rather  to  augment  it. 

3 .  Approach 

An  analytical  approach,  consisting  of  the  development  of  a 
quantitative  model  and  the  determination  of  forms  of  compatible  data,  was 
selected.  An  overview  of  the  approach  is  provided  in  Figure  1. 

The  first  activity  in  Figure  1  lists  benefits  from  improving  rations. 
These  benefits  provide  linkage  between  the  users  (what  they  want),  and 
what  technologies,  through  the  proposals  which  will  pursue  them,  may 
provide.  The  second  initial  activity,  compiling  a  set  of  technology  base 
proposals,  provides  alternative  means  by  which  one  or  more  benefits  may 
occur.  The  list  of  benefits  is  then  used  as  the  basis  for  a  survey  whose 
objective  is  to  determine  the  relative  value  of  each  benefit  to  users. 

The  survey  responses  are  analyzed  to  determine  these  relative  values, 
which  will  subsequently  be  used  as  'weighting  factors'  in  the  final 
cost/benefit  equation. 
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Figure  1.  Overview  of  Prioritization  Process 
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Independently,  a  committee  of  scientists  evaluates  the  proposals, 
estimating  the  likelihood  that  each  proposal  will  produce  at  least  one 
hi'ticilt,  the  probabilities  that  particular  technologies  will  produce  a 
benefit  and  the  likelihood  that  each  benefit  will  be  among  those  produced. 

The  committee's  collective  judgements  are  then  analyzed  to  develop 
aggregate  probabilities  for  every  technology-benef it  combination  applicable 
to  each  proposal. 

In  the  last  major  step  of  the  process,  data  from  the  previous  steps  are 
applied  to  a  weighted  cost/benefit  equation,  to  produce  a  score  for  each 
proposal.  The  magnitude  of  these  scores  determines  each  proposal's  place  in 
a  ranking,  or  prioritization,  of  the  proposals. 

This  type  of  approach  had  several  advantages.  It  is  relatively 
economical,  is  highly  compatible  with  more  objective  forms  of  data 
collection,  and  promotes  clarity  regarding  the  rationale  underlying 
subsequent  analysis.  By  contrast,  alternatives  such  as  soliciting  input 
from  users  by  providing  them  with  an  actual  set  of  technology  base  proposals 
would  be  extremely  expensive,  administratively  burdensome,  and  might  result 
in  incompatible  forms  of  input  (from  simple  ranks  of  the  proposals  to  highly 
qualitative  statements),  thus  making  subsequent  analysis  of  the  user 
community's  collective  preferences  virtually  impossible  to  estimate. 
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II.  Specifics  of  the  Analytical  Methodology 
1.  Oasic  Model 


The  need  to  prioritize  a  set  of  proposals  is  based,  at  least  in  part,  on 
the  recognition  that  resources  are  not  unlimited  and  must  therefore  be 
employed  judiciously.  However,  the  cheapest  proposals  are  not  necessarily 
the  best.  The  cost  of  a  project  should  logically  be  evaluated  in  relation 
to  the  benefits  to  be  derived  from  it.  This  relativity  suggested  a  basic 
'cost/benef i~ '  approach  to  the  prioritization  model. 

All  ben. cit  ;  are  not  necessarily  equal.  A  user  may  attach  enormous 
importance  to  some  benefits,  while  perceiving  relatively  little  utility  in 
others.  To  accommodate  such  priorities  among  benefits,  the  basic 
cost/benefit  approach  was  modified  as  illustrated  in  Equation  1 : 


Benefit  *  Benefit's  Importance  to  User  Equation  1 

Cost 


Since  a  proposal  may  be  expected  to  yield  multiple  benefits,  its  total 
'score'  is  calculated  as  the  sum  of  the  ratios  introduced  in  Equation  1. 
Analogously,  when  there  are  multiple  proposals,  their  total  scores  are 
ranked;  to  produce  a  ’one-to-one'  prioritization. 

2 .  Estimating  Costs 

As  part  of  the  current  program- development  process,  an  estimate  of  a 
proposal's  cost  and  a  general  description  of  its  anticipated  benefits  are 
available.  However,  if  quantitative  evaluations  of  the  proposals  are  to  be 
conducted,  a  more  specific  linkage  between  costs  and  benefits  is  required. 

As  a  first  step,  the  total  funding  of  all  candidate  proposals  is 
calculated.  Each  proposal's  funding  is  compared  to  this  total,  to  produce  a 
'normalized'  cost  estimate  for  that  proposal. 

Proposals  which  are  funded  become  'projects'.  Projects,  in  turn, 
consist  of  one  or  more  'work  units'.  These  work  units  address  differing 
aspects  of  the  overall  project's  objective.  The  application  of  specific 
technologies,  in  conjunction  with  the  project's  goals  and  the  work  unit's 
supporting  objective  and  approach,  constitutes  the  path  through  which 
benefits  attributable  to  that  project  will  ultimately  be  derived.  A  general 
description  of  this  linkage  is  provided  in  Figure  2. 

Next,  a  proposal's  normalized  funding  is  distributed  proportionally 
among  its  work  units.  For  example,  if  a  particular  proposal  requests 
$300,000  out  of  total  amounting  to  $1,000,000,  its  proportion  of  total 
requested  technology  base  resources  is  30%.  If  one  of  its  work 
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Figure  2.  Generic  Description  of  Proposal-Benefit  Relationship 

Note:  Progressing  rightward,  the  subdivisions  (combinations  of  particular 
work  units-technologies-benefits)  in  such  a  network  or  'tree'  diagram  are 
referred  to  as  its  'branches.' 
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units  will  receive  $30,000  (10%  of  the  proposal's  funds),  the  proportion 
attributable  to  this  work  unit  is  (30%)  *  (10%),  or  3%  to  total  technology 
base  resources. 

At  the  work  unit  level,  estimation  of  funds  to  be  spent  pursuing  each 
technology  is  not  currently  required.  In  lieu  of  such  funding 
stipulations,  the  proportion  of  time  or  effort  (i.e.  man-years)  expended 
on  each  technology  must  be  estimated.  Of  course,  the  first  step  in  such  a 
process  is  to  specify  a  collectively  exhaustive  set  of  those 
technologies.  From  this,  a  process  combining  inductive  reasoning  with 
intuitive  insight  is  then  applied. 

For  example,  assume  that  a  particular  work  unit  will  pursue  three 
technologies.  In  lieu  of  any  evidence  to  the  contrary,  they  will  be 
presumed  to  all  contribute  proportionately.  In  the  current  example,  this 
would  amount  to  dividing  their  collective  effort  (100%)  by  the  number  of 
technologies;  that  is,  1/3  or  33  1/3  %  per  technology.  Alternatively, 
some  distinction  between  the  relative  contributions  of  technologies  may  be 
possible.  Continuing  the  example,  suppose  that  one  technology  can  be 
described  as  the  'major  contributor'  to  this  work  unit.  'Major'  can  be 
interpreted  as  anything  over  50%.  Thus,  as  a  minimum,  .51  could  be 
assigned  as  the  estimated  contribution  of  this  technology,  with  each 
member  of  the  residual  subset  assigned  (1- . 51)/(3-l)  or  .245  for  each  of 
the  remaining  two  technologies . 

Care  must  be  exercised  during  this  process,  to  avoid  logical 
contradictions.  In  the  previous  example,  assume  that  two  of  the  three 
technologies  constitute  the  majority  of  effort.  Under  such  circumstances, 
the  proportion  assigned  to  each  majority- technology  would  by  .51/2  or 
.255.  Yet  the  remaining  'minority'  technology  would  have  to  be  evaluated 
as  1-.51  or  .49,  which  is  greater  than  the  proportion  being  individually 
attributed  to  each  majority  technologies.  Since  this  problem  will  only 
arise  when  the  number  of  technologies  in  the  majority-contribution  subset 
is  greater  than  or  equal  to  50%  to  the  total  number  of  technologies,  the 
following  formula  may  be  employed  to  determine  a  minimum  proportion  for 
defining  'majority': 

P  -  S/(S+s)  +  a 
where : 

P  -  proportion  defining  majority  contribution 

S  -  number  of  technologies  belonging  to  the  majority-contribution's  subset 
s  -  number  of  technologies  in  the  residual  subset 

a  -  any  real  number,  proper  fraction  which  satisfies:  1>P  >.5,  ,5>1-P  >0 

Many  other  schemes  for  weighting  technologies  within  a  work  unit  are 
available  in  management  science  and  related  literature,  should  a 
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particular  application  prove  too  demanding  for  the  approaches  just 
discussed.  The  only  absolute  prerequisite  is  that  the  weighting  scheme 
produce  positive -valued  proportions  which  sum  to  one. 

Finally,  the  proportional  cost  of  a  project,  work  unit,  and  the 
estimate  of  effort  applicable  to  a  technology  are  multiplied.  This 
produces  an  estimate  of  cost  of  pursuing  that  technology  in  the  manner 
indicated  by  the  project  and  work  unit's  description. 

3 .  Estimating  Benefits 

The  approach  taken  in  this  FEA  to  the  quantification  of  estimated 
benefits  began  by  recognizing  that  research  is  inherently  a  risk-oriented 
endeavor.  Thus,  a  decision  was  made  to  state  benefits  in  terms  of  the 
probability  that  a  given  type  of  benefit  would  be  forthcoming,  if  a 
technology  were  applied  in  the  manner  suggested  by  a  project  and  work 
unit's  description.  In  terms  of  classical  statistics,  this  is  represented 
by: 


P(  p  and  w  and  t  and  b  )  where:  p  -  project 

w  -  work  unit 
t  -  technology 
b  -  benefit 

Figure  2  demonstrated  that  work  units  are  more  directly  linked  to 
benefits  than  projects.  Therefore,  probabilities  will  henceforth  be 
stated  in  terms  of  work  units.  Note  that  this  provides  the  flexibility  to 
prioritize  the  work  units  themselves,  while  preserving  the  capability  to 
rank  projects,  since  the  value  of  a  project  can  be  interpreted  as  the  sum 
of  the  cost/benefit  ratios  of  its  work  units.  This  interpretation  reduces 
the  objective  to: 

P(  w  and  t  and  b  ) 

Table  1  provides  a  generalized  illustration  of  these  probabilistic 
relationships.  The  cells  of  this  table's  matrix  correspond  to  the  joint 
probability  introduced  above,  within  the  context  of  a  specific  work  unit, 
technology  and  benefit. 

Theoretically,  there  is  no  reason  why  evaluators  could  not  specify 
their  judgements  on  a  cell-by-cell  basis.  However,  such  judgements  would 
have  to  be  made  within  the  context  of  the  work  unit's  objective  and 
approach.  This  means  such  judgements  are  conditional  probabilities,  based 
upon  an  implicit  judgement  about  the  work  unit  as  a  whole.  One  potential 
problem  is  that  when  dealing  with  such  large  numbers  of  estimates,  the 
same  technology -benefit  combinations  may  receive  similar  evaluations, 
without  allowing  for  the  fact  that  they  appear  in  work  units  having 
substantially  different  overall  probabilities.  Note  that  if  there  are  'm' 
work  units,  'n'  technologies,  and  'i'  benefits,  the  gross  number  of  cells 
is  the  product  m*n*i.  As  these  numbers  become  larger,  the  administrative 
burden  could  become  staggering.  This  administrative  consideration,  in 
turn,  could  result  in  the  reluctance  of  qualified  evaluators  to 
participate  in  the  process. 
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Table  1.  Generic  Work  Unit-Technology-Benefit  Matrix 


Work  Unit  x:  P(  w  ) 


P(t  I  w) 


P(b  I  w)  Technology  #1  Technology  #2  ***  n  Technology 

BENEFITS _ th _ 

I  I  II  I 

I  P(  w  and  t  and  b  )  I  P(  w  and  t  and  b  )  I  I  P(  w  and  t  and  b  )  I 

a:  I  x  1  al  x  2  all  x  n  al 

I  I  II  I 


I  I  II  I 

I  P(  w  and  t  and  b  )  I  P(  w  and  t  and  b  )  I  I  P(  w  and  t  and  b  )  I 

b :  I  x  1  bl  x  2  bll  x  n  bl 

I _ I _ I  I _ I 

* 

* 

* 


I  I  II  I 

I  P(  w  and  t  and  b  )  I  P(  w  and  t  and  b  )  I  I  P(  w  and  t  and  b  )  I 

k:I  x  1  kl  x  2  kll  x  n  kl 
I  I  II  I 
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An  alternative  approach  taken  by  this  FEA  was  to  define  the  cell 
values  through  an  extension  of  classical  statistical  calculations 
regarding  joint  probability.  This  defined  a  cell  value  as  follows: 

P(  w  and  t  and  b)  —  P(w)  P(t  I  w)  P(b  I  t  and  w)  Equation  Two 

where:  w  -  work  unit,  t  -  technology,  b  -  benefit 

P(w)  -  probability  that  a  work  unit  will  produce  at  least  one  benefit 

P(t  I  w)  -  probability  that  a  technology  produced  the  benefit,  given  that 
a  benefit  occurs 

P(b  I  t  and  w)  -  probability  that  a  particular  benefit  will  be  among  those 
which  occur,  given  that  at  least  one  benefit  has  occurred  and  a  particular 
technology  produced  it. 

The  first  two  factors  of  Equation  Two  can  be  solicited  directly  from 
each  evaluator.  However,  obtaining  values  for  P(b  I  t  and  w)  would  be  as 
administratively  burdensome  as  the  cell-by-cell  approach.  The  compromise 
was  to  approximate  this  value  using  P(b  I  w) ;  an  estimate  which  required 
only  one  response  per  benefit.  Hence,  the  approximation  formula  employed 
in  the  FEA  process  was : 

P(  w  and  t  and  b)  P(w)  P(t  I  w)  P(b  I  w)  Equation  Three 

where:  P(b  I  w)  -  probability  that  a  particular  benefit  will  occur,  given 
that  at  the  work  unit  produces  at  least  one  benefit 

The  administrative  expedience  of  this  approach  had  its  disadvantages. 
However,  certain  considerations  must  be  taken  into  account  before  arriving 
at  a  final  judgement  concerning  the  reasonability  of  employing  Equation 
Three.  First,  since  equations  two  and  three  are  otherwise  identical,  the 
difference  between  the  two  equations'  last  factor  constitutes  the  only 
source  of  inter-equation  error.  The  maximum  potential  error  in 
P(w  and  t  and  b)  will  occur  when  the  other  two  factors  are  at  their 
maximum  and  P(b  I  w)  is  either  very  large  or  very  small.  Secondly,  the 
value  used  in  the  equation  is  arrived  at  by  computing  a  'consensus' 
estimate  from  the  individual  responses  of  multiple  evaluators.  Hence,  in 
order  for  p(b  I  w)  to  be  a  poor  approximation  of  P(b  I  t  and  w) ,  the 
majority  of  the  evaluators  must  have  made  the  same  error,  and  done  so 
consistently  (e.g.  all  over-estimation,  by  a  substantial  magnitude). 
Lastly,  the  consensus  estimate  employed  was  the  median,  rather  than  the 
mean,  which  helps  minimize  the  effect  of  extreme  judgements  which  are  in 
the  minority. 

The  principal  administrative  benefit  of  this  approach  is  that  it 
reduces  the  number  of  responses  from  m*n*i  to  m*(n+i+l) .  However,  as  will 
be  seen  in  Section  III,  the  required  number  of  responses  can,  in  actual 
practice,  be  reduced  still  further. 
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4 .  Estimating  User  Priorities 


The  approach  chosen  for  this  FEA  was  that  proposed  by  Saaty,  as  part 
of  his  'Analytic  Hierarchy  Process'.  Basically,  this  entails  soliciting 
data  in  the  form  of  paired  comparisons,  while  requiring  that  the  responses 
be  quantified  in  relation  to  a  9-point  hedonic  scale: 

1 . 3 . 5 . 7 . 9 

Equality  Slightly  Moderately  Substantially  Absolutely 

The  elements  being  compared  are  called  'characteristics';  the  traits 
or  qualities  which  are  to  be  prioritized.  In  relation  to  the  FEA,  such 
characteristics  are  what  have  previously  been  referred  to  as  benefits; 
i.e.  those  traits  or  qualities  of  rations  which  users  are  likely  to  deem 
desirable . 

For  example,  military  personnel  have  historically  considered  the 
reduction  of  a  ration's  weight  and  volume  to  be  important  benefits  of 
research.  If  a  respondent  considers  these  benefits  to  be  equal,  the 
applicable  response  is  symbolized  by  the  number  '1'.  However,  if  he 
considers  reducing  a  ration's  weight  to  be  'substantially'  more  important 
than  reducing  its  volume,  the  number  '7'  must  be  associated  with  'weight', 
when  responding  to  that  pair  of  alternatives. 

Once  such  data  has  been  solicited,  it  is  analyzed  as  depicted  in  Table 

2: 


Table  2.  Calculating  User  Priorities 


Example 

One : 

Weight 

Volume 

Row 

Sums 

Row  Sum/Matrix  -  Priority 

Weight 

I  1  I 

1  I 

2 

Weight:  2  /  4  -  .5 

Volume 

I . I- 

III 

. I 

1  I 

2 

Volume:  2  /  4  -  .5 

Example 

Two: 

Weight 

Matrix:  4 

Row 

Volume  Sums 

Row  Sum/Matrix  -  Priority 

Weight 

III 

7  I 

8 

Weight:  8  /  9.143  -  .875 

Volume 

I . I- 

I  .143  I 

. I 

1  I 

1.143 

Volume:  1.143  /  9.143  -  .125 

Matrix: 

9.143 

Regarding  Table  2,  note  that  cells  corresponding  to  the  intersection 
of  the  same  characteristic  (e.g.  row  'weight',  column  'weight')  are 
automatically  presumed  to  equal  one;  signifying  that  a  characteristic  is 
always  'equal'  to  itself.  Value  in  cells  correspond  to  judgements  made 
about  them.  For  example,  in  Example  Two,  weight  was  judged  to  be 
'substantially'  more  important  than  volume.  Thus,  the  number  '7'  was 
entered  in  cell  (weight,  volume).  The  inverse  relationship  is  inferred 
for  cell  (volume,  weight).  Thus,  1/7  or  .143  occupies  that  cell.  The 
cell  values  are  summed  rowwise,  and  a  total  for  all  cells  is  computed. 
The  right  side  of  Table  2  shows  how  these  computations  are  then  used  to 
develop  priority  values  for  each  benefit. 
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Although  the  previous  discussion  highlights  the  basic  process  of 
calculating  user  priorities,  it  must  be  acknowledged  that  not  all  users 
have  the  same  priorities.  The  procedure  selected  to  deal  with  this 
expected  diversity  was  Hartigan's  K-Means  approach.  This  form  of 
multi-variate  clustering  algorithm  partitions  an  initial  set  of  data  into 
the  best  statistical  'fit'  (considering  multiple  criteria  of 
classification);  given  a  specified  number  of  groups,  'K' .  For  example,  if 
'K'  is  set  equal  to  two,  the  algorithm  will  partition  all  responses  into 
one  of  two  subsets  it  will  create,  according  to  the  overall  similarity 
among  responses. 

In  conjunction  with  the  FEA  process,  the  next  step  (after  clustering) 
is  to  develop  aggregate  responses  for  each  cluster.  The  reader  should 
keep  in  mind  that  although  the  clustering  phase  partitions  the  data  into 
their  'best  fit',  this  does  not  mean  that  the  cluster  now  contains 
identical  responses  to  any  given  question.  Some  dispersion  usually  exists 
among  each  set  of  responses.  Thus,  a  'typifying'  response  must  be 
developed.  The  median  is  the  preferred  measure  of  central  tendency  to 
accomplish  this  goal,  because  it  is  relatively  unaffected  by  extreme 
responses  which  are  in  the  minority. 

Finally,  the  proportion  of  respondents  to  the  total  sample  size 
corresponding  to  each  cluster  is  calculated.  After  the  group- responses  of 
each  of  the  two  clusters  has  been  calculated  by  the  Saaty  process 
discussed  earlier,  the  sample-size  proportions  are  used  to  adjust  the 
respective  user - evaluations .  This  has  the  effect  of  limiting  the 
influence  of  each  set  of  priorities  on  the  overall  cost/benefit  equation 
according  to  the  proportion  of  the  sample  each  set  of  priorities 
represents . 

Since  there  are  a  number  of  steps  in  the  FEA  process,  and  since  some 
of  these  procedures  may  be  unfamiliar  to  the  reader,  this  section  will 
conclude  by  presenting  a  succinct  hypothetical  example  of  the  overall 
process  in  a  start-to-f inish  manner. 

The  top  section  of  Table  3  depicts  two  cost  estimations  of  technology 
'a'.  The  first  example  (work  unit  x)  was  previously  introduced  in  Section 
II. 2,  where  it  was  shown  that  work  unit  x  requests  3%  of  all  resources 
requested,  and  technology  'a'  will  consume  25%  of  these  resources;  or 
.0075  of  total  requested  resources.  Analogously,  work  unit  y  also  will 
receive  3%  of  total  resources,  but  plans  to  spend  50%  pursuing  technology 
a;  or  .015  of  total  resources. 

The  next  section  describes  the  'raw'  (prior  to  including 
user-priorities)  cost/benefit  ratios,  in  relation  to  the  benefit  'reducing 
ration  weight'.  In  practice,  these  estimates  would  have  come  from 
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Table  3.  Example  of  the  FEA  Prioritization  Process 


Cost 

of  work 

unit- technology  'branch': 

Work  Unit 

Technology  'a' 

Technology  'a 

Resources 

*  %  of  Work  Unit  - 

Resources 

Work 

Unit  x: 

3% 

25% 

.0075 

Work 

Unit  y: 

3% 

50% 

.015 

'Raw'  (unadjusted)  Cost/Benefit  Relationship: 

Branch ' s 

Work  Benefit 

Branch's 

Branch's 

Raw 

Unit  Type 

Benefit 

/ 

Resources 

-  Score 

x:  Weight 

.5 

.0075 

66.67 

x:  Volume 

.2 

.0075 

26.67 

Total 

Score:  93.34 

y:  Weight 

.5 

.015 

33.33 

y:  Volume 

.7 

.015 

46.67 

Total  Score:  80.00 


User-Adjusted  Cost/Benefit  Ratios: 


Work 

Benefit 

Branch's 

User  l's 

Branch ' s 

Adjusted 

Unit 

Type 

Benefit  * 

Priority 

/ 

Resources  - 

Score 

x: 

Weight 

.5 

.5 

.0075 

33.33 

x: 

Volume 

.2 

.5 

.0075 

Total  Score 

13.33 
:  46.67 

y: 

Weight 

.5 

.5 

.015 

16.67 

y: 

Volume 

.7 

.5 

.015 

Total  Score 

23.33 
:  40.00 

Work 

Benefit 

Branch ' s 

User  2's 

Branch's 

Adjusted 

Unit 

TyPe 

Benefit  * 

Priority 

/ 

Resources  - 

Score 

x : 

Weight 

.5 

.875 

.0075 

58.33 

x : 

Volume 

.2 

.125 

.0075 

Total  Score 

3.33 
:  61.67 

y: 

Weight 

.5 

.875 

.015 

29.17 

y: 

Volume 

.7 

.125 

.015 

Total  Score 

5.83 
:  35.00 

Adjustment  for  Sample  Size  Proportion: 

Before  Sample  Size  Adjustment  %  of  After  Sample  Size  Adjustment 


Work  Unit  x 

Work  unit  y 

Sample 

Work  Unit  x 

Work  unit  y 

User  #1: 

46.67 

40 

90 

42.00 

36 . 00 

User  #2: 

61.67 

35 

10 

6.17 

3.5 

Total  : 

108.34 

75 

48.17 

39.5 
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technological  evaluations,  as  discussed  in  Section  II. 3  Under  the  column 
labeled  'Branch's  Raw  Score'  note  that  work  unit  x's  score  for  the  row 
'weight'  is  exactly  double  that  of  work  unit  y.  This  occurs  because 
although  they  produce  equal  benefits,  work  unit  x  does  so  with  resources 
half  the  size  of  work  unit  y's.  The  total  scores  are  relatively  close 
because  work  unit  y's  anticipated  benefit  'volume'  is  substantially  larger 
than  that  of  work  unit  x,  which  compensates  somewhat  for  its  relative 
inefficiency  in  regard  to  weight. 

The  section  entitled  'User-Adjusted  Cost/Benefit  Ratios’  is  further 
divided  into  two  subordinate  sections.  The  first  reflects  the  influence 
of  User  #1  on  the  raw  scores .  The  second  subsection  demonstrates  the 
effect  of  User  #2's  priorities  on  the  raw  scores.  These  user-priorities 
are  the  same  as  those  in  Table  2,  to  facilitate  the  reader's  understanding 
of  how  such  priorities  were  arrived  at,  and  how  they  fit  into  the  overall 
process.  Compare  the  total  scores  for  the  two  work  units  which  appear  in 
the  first  subsection  to  those  of  the  latter  subsection.  The  scores 
resultant  from  User  #l's  priorities  are  proportionately  identical  to  those 
of  the  raw  totals.  This  occurs  because  user  1  does  not  favor  either 
benefit.  However,  since  User  #2  has  expressed  a  substantial  priority 
regarding  the  reduction  of  weight  rather  than  volume,  work  unit  x's 
relative  efficiency  in  expending  resources  to  achieve  the  former  benefit 
contributes  highly  to  its  total  score. 

The  final  subsection  of  Table  3  demonstrates  the  effect  of 
hypothetically  clustering.  User  #l's  priorities  are  assumed  to  represent 
90%  of  the  total  sample,  while  the  priorities  of  User  #2  account  for  the 
remaining  10%.  The  left  side  shows  the  sums  of  scores  from  both  users 
(taken  directly  from  the  previous  subsection).  The  right  shows  the  effect 
of  further  adjusting  these  scores  by  the  proportion  of  sample  size  each 
priority  represents.  For  example,  in  the  row  labeled  'User  #1',  the 
previous  scores  are  multiplied  by  .9,  to  produce  adjusted  scores  for  Work 
Units  1  and  2  of  42  and  36,  respectively.  Although  in  this  example,  work 
unit  x  retains  its  superior  score,  note  the  substantial  drop  in  magnitude 
between  scores.  This  indicates  that  when  the  priorities  of  the  second 
group  are  recognized  as  being  an  extreme  minority  point  of  view,  the 
implied  preference  of  the  user  community,  as  a  whole,  for  work  unit  x  is 
not  substantial. 
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III.  Prioritization  Exercise 


1 .  Objective 

Although  the  methodology  presented  in  Section  II  proposed  a  logical 
series  of  steps  to  accomplish  the  FEA's  stated  objectives,  it  was  felt 
that  the  practicality  of  implementing  the  approach  under  real-world 
conditions  would  add  to  its  credibility.  Therefore,  the  objective  of  the 
prioritization  exercise  was  to  demonstrate  its  execution  under  the  most 
realistic  conditions  possible. 

2 .  Scope 

a.  Orientation  Away  From  Immediate/Near-Term  Perspective 

As  previously  addressed  (see  Section  I.I.),  the  technology  base  is 
primarily  concerned  with  military  food  research  which  is  not  as  closely 
linked  to  immediate  needs  as  other  segments  of  the  DOD  Food  Program.  As 
such,  it  was  deemed  desirable  to  focus  the  exercise  toward  a  'long  term' 
perspective.  That  is,  to  recruit  representatives  from  the  user  community 
who  could  orient  their  thinking  away  from  immediate  concerns  regarding 
military  subsistence.  Likewise,  technological  evaluators  would  have  to  be 
recruited  who  possessed  both  the  expertise  and  willingness  to  evaluate 
proposals  developed  from  'notional  systems';  highly  theoretical  concepts 
based  on  attempts  to  anticipate  user  requirements  of  the  future. 

b.  DOD-Wide  Perspective 

Since  all  military  services  utilize  subsistence-related  items 
developed  by  Natick,  it  was  felt  that  all  should  be  given  an  opportunity 
to  participate  in  this  exercise.  In  conjunction  with  the  long-term 
perspective  previously  discussed,  it  was  decided  to  recruit  these  ’user 
representatives'  from  those  organizations  within  each  military  service 
which  were  anticipated  to  be  the  most  capable  of  taking  the  'long  term' 
view.  Hence,  organizations  involved  in  military  planning  were  targeted. 
Appendix  A  lists  these  organizations. 

c.  Concentration  on  the  'Individual'  Mode  of  Feeding 

A  DOD-wide  survey,  alone,  could  be  anticipated  to  require  considerable 
time  to  develop,  administer,  and  analyze.  Additionally,  a  set  of 
proposals  would  have  to  be  selected  and  evaluated.  Limiting  the  exercise 
to  consider  only  'individual'  forms  of  subsistence  (those  carried  by 
troops,  as  opposed  to  subsistence  systems  where  food  is  prepared  in  bulk) 
could  vastly  simplify  matters.  Since  this  restricted  scope  could  help 
keep  resource  expenditures  within  budget,  while  still  achieving  the 
objective  of  the  exercise,  a  decision  was  made  to  do  so. 

3 .  Development  of  the  Set  of  Benefits 

Benefits  provide  the  linkage  between  what  users  can  be  anticipated  to 
want  and  what  technologies  (i.e.  the  proposals  which  pursue  them)  may 
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provide.  For  this  reason,  considerable  time  and  thought  was  invested  in 
the  development  of  the  set  of  benefits.  Guidance  and  input  was  sought 
from  a  number  of  vantage  points,  including  behavioral  science,  food 
science  and  operations  research,  as  well  as  that  of  active  duty  military 
personnel.  The  final  set  of  benefits  and  their  definitions  represents 
both  compromise  and  consensus  among  the  parties  involved  in  its 
development.  The  set  is  not  all-inclusive.  However,  it  was  felt  that  the 
set  did  constitute  a  good  core  representation  of  the  types  of  benefits  the 
user  community,  as  a  whole,  would  be  likely  to  find  most  relevant. 
Moreover,  behavioral  specialists  and  others  having  expertise  in 
survey -development  counseled  against  a  more  expansive  set,  since, 
historically,  there  appears  to  be  a  high  inverse  correlation  between  a 
survey's  size  and  the  willingness  of  respondents  to  participate  in  it. 

The  final  set  of  benefits  and  their  definitions  are  shown  in  Table  4. 

4 .  Administration  of  the  User  Survey 

The  user  community's  response  to  the  survey  was  extremely  positive. 

Of  the  approximately  500  surveys  sent  out,  about  90%  were  returned 
completed.  Of  these,  only  about  3%  were  incorrectly  completed.  A  number 
of  factors  contributed  to  this.  Mailing  the  surveys  allowed  a  large 
number  of  them  to  be  disseminated  economically .  In  addition  to  the  care 
exercised  in  the  development  of  the  set  of  benefits,  considerable 
attention  was  paid  to  the  development  of  the  survey  instrument  itself. 

The  survey  consisted  of  three  sections.  The  first  briefly  reviewed 
the  objectives  of  the  survey  and  the  FEA  in  general.  This  served  to 
immediately  orient  the  respondents'  thinking  away  from  the  near  term.  The 
second  section  contained  detailed  instructions.  The  last  section,  which 
contained  the  actual  benefits  to  be  compared,  was  further  divided  into 
four  subsections.  The  reason  for  this  can  be  seen  by  recognizing  that  the 
number  of  required  paired  comparisons  is  exponentially  related  to  the 
number  of  benefits;  that  is: 


Required  Paired  Comparisons  -  (#  of  benefits) *2  -(#  of  benefits) 

2 

Thus,  the  13  benefits  constituting  the  finalized  set  would  have 
required  78  paired  comparisons.  A  compromise  was  reached  by  making  use  of 
the  fact  that  some  of  the  13  characteristics  are  inherently  related.  For 
example,  ease  of  preparation,  as  a  general  concept,  is  to  a  large  degree 
defined  by  ease  of  opening  and  ease  of  heating.  Likewise,  durability, 
decontamination  and  disposal  are  all  attributes  of  packaging,  while 
variety,  taste  and  recognizability  are  all  'acceptance'  characteristics. 
Hence,  preparation  (as  a  general  concept),  packaging,  and  acceptability 
were  initially  substituted  for  their  respective  subsets.  This  reduced  the 
set  of  'major'  benefits  to  7,  and  thus  the  initial  number  of  required 
paired  comparisons  to  21.  Each  of  the  succeeding  subsections  addressed 
one  of  the  three  'subordinate'  subsets  of  benefits.  Each  subset  required 
three  paired  comparisons;  for  a  total  of  nine.  This  brought  the  total 
comparisons  required  by  the  survey  to  30,  or  about  40%  of  what  would  have 
otherwise  been  required. 
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Table  4.  Set  of  Benefits 


1.  Reducing  Weight:  This  benefit  refers  to  the  total  weight  of  a  ration; 
that  is,  the  combined  weight  of  its  food  and  packaging. 

2.  Reducing  Volume:  As  with  'reducing  weight’,  the  meaning  of  this  benefit 
relates  to  total  volume,  including  the  influence  on  volume  of  both  the 
food  and  its  packaging. 

3.  Maintaining  Strength  and  Alertness  (S  &  A) :  It  is  recognized  that  all 
combatants  have  a  basic  need  to  maintain  their  physical  strength,  as  well 
as  to  remain  mentally  alert.  Assessments  of  this  benefit  seek  an 
indication  of  its  relative  importance,  within  the  context  of  the  specific 
operational  conditions  applicable  to  a  particular  user-group. 

4.  Satiety:  As  was  the  case  with  S  &  A,  it  is  recognized  that  the 
alleviation  of  hunger  is  a  basic  motivation  for  eating.  Again,  the 
judgement  being  sought  is  one  of  relativity,  in  relation  to  the  other 
benefits . 

5.  Increasing  Ease  of  Preparation:  'Preparation'  is  defined  as  all 
activities  which  are  normal  and  necessary  to  bring  o  ration  from  its  fully 
packaged  state,  to  a  condition  in  which  it  is  ready  for  immediate 
consumption.  However,  ease  of  preparation  is  a  very  general  term. 
Therefore,  this  benefit  is  followed  by  three  'subordinate'  benefits,  each 
of  which  further  defines  a  specific  aspect  involved  in  making  rations 
easier  to  prepare: 

5a.  Reducing  Preparation  Time/Steps,  in  General:  Preparing  individual 
rations  may  require  a  number  of  activities,  such  as  mixing,  stirring,  etc. 
In  evaluating  this  benefit,  its  importance  in  relation  to  the  remaining 
two  specific  aspects  of  easy  preparation,  ease  of  opening  and  heating, 
should  be  considered. 

5b.  Increasing  Ease  of  Opening:  Although  the  meaning  of  this  benefit  may 
appear  self-evident,  the  operational  circumstances  under  which  a  ration 
will  be  used  must  be  taken  into  consideration.  For  example,  during  cold 
weather  operations,  during  which  gloves  will  probably  be  worn,  the 
importance  of  easy  opening  may  be  expected  to  increase. 

5c.  Ease  of  Heating:  The  availability  of  required  time  to  a  particular 
user-group,  as  well  as  the  logistical  impact  and  battlefield  practicality 
of  providing  fuel,  equipment,  etc.  must  be  taken  into  account  when 
assessing  the  relative  value  of  this  benefit. 

6.  Improving  Packaging:  Like  preparation,  packaging  has  been  further 
defined  by  the  following  three  subordinate  benefits: 

6a.  Increasing  Packaging  Durability:  The  primary  factor  to  be  considered 
in  evaluating  this  benefit  is  the  degree  of  rough  handling  which  can  be 
anticipated . 
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Table  4.  Set  of  Benefits  (Cont'd.) 


6b.  Increasing  Ease  of  Packaging's  Decontamination:  Factors  effecting  the 
relative  importance  of  this  benefit  are  the  likelihood  of  the  need  to 
decontaminate  individual  rations ,  the  speed  and  ease  with  which  standard 
decontamination  procedures  may  be  applied,  and  the  applicability  of 
commonly  available  decontamination  materials  (e.g.  soap  and  water). 

6c.  Increasing  Ease  of  Disposal:  The  importance  of  avoiding  a  'trash 
signature'  must  be  considered  in  evaluating  this  benefit.  For  example, 
the  mission  of  reconnaissance  forces  are  inherently  clandestine, 
necessitating  that  the  risk  of  signatures  of  any  kind  be  minimized. 
Alternatively,  support  forces  may  evaluate  disposal's  importance  in  terms 
of  the  time,  steps  and/or  equipment  (e.g.,  entrenching  tool)  required  to 
deal  with  refuse  from  rations. 

7.  Improving  Acceptability:  This  benefit  addresses  the  importance  of  the 
qualitative  aspects  of  food.  Again,  it  is  further  defined  by  subordinate 
benefits : 

7a.  Maintaining  Recognizability :  Some  food  items  currently  available  in 
individual  rations  such  as  the  Meal,  Ready- to-Eat  do  not  look  the  same  as 
'everyday'  food  (e.g.  freeze-dried,  compressed  meat  bars).  The  foods  in 
future  rations  may  look  even  less  familiar,  as  a  result  of  technological 
applications  intended  to  achieve  other  benefits.  Therefore,  the 
importance  of  food's  recognizability,  relative  to  other  anticipated 
benefits,  must  be  estimated. 

7b.  Improving  Taste:  Future  rations'  foods  could  have  a  totally  unfamiliar 
taste,  taste  very  bland,  or  otherwise  differ  from  the  that  of  everyday 
subsistence.  Alternatively,  very  'tasty'  items  may  be  counterproductive 
to  the  pursuit  of  other  benefits.  Thus,  the  relative  importance  of  food's 
taste  must  be  considered. 

7c.  Increasing  Variety:  The  number  of  different  menus,  as  well  as  the 
number  of  different  components  within  each  menu,  can  collectively  be 
thought  of  as  a  ration  system's  variety.  Once  again,  its  relative 
importance  to  a  particular  user-group  may  depend  upon  the  battlefield 
conditions  under  which  it  is  intended  to  be  eaten. 


17 


Table  5  shows  the  distribution  of  the  major  benefits'  initial  scores 
to  their  subordinate  benefits;  to  arrive  at  priority  values  for  the  final 
set  of  thirteen  benefits.  For  example,  preparation  initially  attained  a 
relative  score  of  .107,  in  comparison  to  the  other  major  benefits.  Next, 
an  estimate  is  made  to  distinguish  between  the  respondents  attraction 
toward  preparation  as  a  general  concept,  as  opposed  to  the  specific 
attributes  of  easy  opening  or  heating.  Thus,  .107  is  multiplied, 
respectively ,  by  .2,. 3,  and  .5,  to  arrive  at  a  distribution  of  this  major 
benefit's  score  to  its  subordinates.  The  products  (.021,. 032,  and  .054) 
associated  with  these  subordinate  benefits  appear  in  the  finalized 
priorities  on  the  right  of  Table  5. 

TABLE  5.  Example  of  Distributing  Major  Benefits'  Priorities 

to  their  Subordinates 


Maj  or 

Major 

Sub-  ! 

Sub- 

Final 

Final 

Characteristics 

Values 

Characteristics  Values 

Characteristics 

Values 

Weight 

.15 

Weight 

.  15 

Volume 

.  12 

Volume 

.12 

S  &  A 

.21 

S  &  A 

.21 

Hunger 

.20 

Hunger 

.20 

1-  -> 

(general)  Prep 

.200 

(general)  Prep 

.021 

Preparation 

.107---: 

I-  -> 

Opening 

.  300 

Opening 

.032 

I--> 

Heating 

.500 

Heating 

.054 

I--> 

Durability 

.747 

Durability 

.072 

Packaging 

.096---: 

[--> 

Decontamination 

.250 

Decontamination 

.024 

I--> 

Disposal 

.003 

Disposal 

.0003 

[--> 

Taste 

.450 

Taste 

.053 

Acceptability 

.117---: 

[--> 

Variety 

.367 

Variety 

.043 

i --> 

Recognizability 

.  183 

Recognizability 

.021 
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5.  Administration  of  the  Technical  Evaluations 


The  first  step  in  this  activity  was  to  select  a  set  of  candidate 
proposals.  It  was  felt  that  using  the  FEA  process  to  evaluate  actual 
proposals  would  be  premature.  Therefore,  a  set  of  'surrogate'  proposals  , 
compiled  from  existing  work  units,  was  selected.  Another  advantage  of 
this  decision  was  the  immediate  availability  of  descriptions  (objective 
and  scope)  and  funding  levels  for  these  candidates.  This  decision,  in 
turn,  led  to  a  decision  to  employ  not  only  technology  base  projects,  but 
existing  MSR's  as  well.  It  was  felt  that  if  a  reasonably  flexible 
interpretation  of  the  term  'technology'  was  applied,  these  types  of 
proposals  could  also  be  included,  since  they  too  utilize  technological 
means  to  pursue  their  near-term  objectives.  The  advantage  being  that  the 
FEA  process  might  subsequently  be  utilized  not  only  to  influence  the 
technology  base,  but  also  the  entire  DOD  Food  Program.  The  finalized  set 
of  work  units  which  served  as  surrogate  proposals  is  listed  in  Table  6: 

Table  6.  Work  Units  Employed  in  the  Technological  Evaluation  Exercise 

PROPOSAL 
NUMBER 


FTB1259 

FTB1260 

FTB1262 

FTB1265 

FTB1266 

FTB1270 

FTB1271 

FTB1276 

FTB1279 

MSR1426 

MSR1449 

MSR1485 

MSR1486 

MSR1497 

MSR1498 

MSR1509 

MSR1511 


TITLE 


Universal  Nutrient  Dense  Components 

High  Efficiency  Preservation  Technologies 

Energy  Optimization  in  Food  Service 

Human  Factors  of  Ration  Consumption 

Sensory  Engineering  of  Rations 

Quality  Retention  &  Nutrient  Bioavailability 

Attainments  and  Validation  of  Microbial  Control 

Thermostabilized  Meals  for  Combat  Vehicle  Crews 

Development  of  Polymeric  Packaging  Materials 

Design  of  USAF/USN  Flight  Feeding  System 

Nutritional  Sustainment  Module 

Thermostabilized  Meals  for  Remote  Areas 

Develop  Compact/Springback  Bread 

Lowfat  Instantized  Operational  Ration  Milk 

New  Generation  Survival  Rations 

Food  Service  for  Rail  Garrison  Mobile  Missiles 

Self-heating  for  the  Year  2000 
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The  specification  of  technologies  used  in  the  technological  evaluation 
exercise  appear  in  Table  7.  Note  that  the  30  technologies  were  placed 
into  specific  categories,  which  were  defined  for  the  evaluators.  This 
approach  was  undertaken,  despite  the  high  likelihood  that  qualified 
evaluators  should  already  be  familiar  with  most  or  all  of  the 
technologies,  to  ensure  that  they  considered  each  technology  within  the 
intended  context  provided  under  the  category's  description. 

Moreover,  during  preparation  of  the  survey  material,  it  was  noted  that 
not  every  technology  listed  in  Table  7  applied  to  every  work  unit.  For 
example,  the  technologies  of  the  'protection'  category  were  deemed  to  have 
little  relevance  to  work  unit  FTB1259  (Universal  Nutrient  Dense 
Components) ,  since  this  effort  was  not  directly  related  to  packaging.  By 
the  same  logic,  the  packaging  benefits  were  deemed  equally  inappropriate. 
The  effect  of  such  eliminations  was  to  reduce  the  administrative  burden  on 
respondents  still  further,  in  instances  where  either  P(t  I  w)  or  P(b  I  w) 
was  anticipated  to  be  virtually  zero.  In  the  case  of  FTB1259,  22 
technologies  and  9  benefits  were  ultimately  presented  to  the  evaluators. 

During  the  evaluation  process,  itself,  one  further  step  was  taken. 

The  applicability  of  a  technology  and  a  benefit  to  a  work  unit  does  not 
necessarily  imply  a  relationship  between  all  technologies  and  benefits. 
Continuing  the  example,  although  the  technology  'preservatives'  and  the 
benefit  of  reducing  a  ration's  weight  were  both  deemed  appropriate  to  the 
evaluation  of  FTB1259,  that  particular  combination  of  technology-benefit 
was  not.  While  this  recognition  provided  no  administrative  advantage 
during  the  evaluation  process,  such  irrelevant  combinations  were  later 
filtered  out  electronically,  during  the  analysis  of  the  evaluations. 

The  selection  of  participants  included  both  Natick  scientists  as  well 
as  technologists  from  the  civilian  community.  In  these  latter  instances, 
the  'recruiter'  (Battelle  Corp,  under  an  Army  Research  Office  contract  to 
support  this  FEA)  provided  a  list  of  potential  evaluators  to  Natick,  along 
with  evidence  of  each  candidate's  technological  credentials.  From  this 
list,  four  out-of -house  evaluators  were  selected  and  added  to  the  six 
Natick  participants,  for  a  total  of  ten  members  composing  the  finalized 
evaluation  team. 
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Table  7.  Technologies  Applied  to  Individual  Ration  Development 


Category 


Process 


Design 


Protection 


Assessment 


_ Description _ 

Process  and  techniques  by  which 
principles  of  food  engineering  and 
process  control  are  applied  to 
produce  foods  stabilized  by  adding 
preservatives  and/or  by  heating, 
drying,  extruding/infusing,  and 
irradiating  to  retard  or  destroy 
microbial  contaminants. 


Procedures  and  approaches  by  which 
principles  of  nutrition,  food 
formulation,  sensory  attribute 
control,  and  consumer  marketing  are 
applied  to  produce  rations  of 
suitable  nutrition,  acceptance,  and 
modularity. 


Methods  and  processes  by  which  the 
principles  of  material  science  and 
physical  chemistry  are  applied  to 
the  development  of  coatings,  films, 
flexible  and  rigid  containers ,  and 
odified  gaseous  atmospheres  to 
control  the  transmission  of 
molecular  components  between  the 
ration  and  its  environment  and  to 
protect  the  ration  from  physical 
abuse . 


Procedures  and  methods  by  which  the 
principles  of  quality  assurance, 
human  behavior,  and  operations 
research  are  applied  to  assess  the 
safety,  bioavailability  of  proteins/ 
carbohydrates,  enhancement  of 
physical  and  mental  performance, 
retention  of  quality,  and  overall 
system  effectiveness  of  all 
operational  rations. 


Technologies 

Preservatives 

Thermoprocessing 

Dehydration 

Extrusion/Infusion 

Nutritional  Engineering 
Ingredient  Formulation 
Sensory  Engineering 
Consumer  Marketing 
Modularization 

Coatings 

Films 

Containers 
Gaseous  Overwrap 


Safety  Factors 
Nutrient  Availability 
Performance  Enhancement 
Quality  Limitations 
Systems  Effectiveness 
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Table  7.  Technologies  Applied  to  Individual  Ration  Development  (Cont'd) 


Category 


Human 

Factors 


Logistics 


Equipment 


Biotechnology 


Description 


Methods  by  which  the  principles  of 
human  factors  engineering  and 
biodynamics  are  applied  in 
developing  artificial  intelligence 
(AI)  and  optimization  programs 
for  selecting  proper  ration 
components  convenient- to-handle 
rations,  efficient  and  safe  heating 
/cooling  devices,  and  package 
disposal  procedures. 


Procedures  and  practices  by  which 
the  principles  of  operations 
research  and  management  science  are 
applied  in  devising  systems  for 
tailoring  modules  into  scenario 
-specific  rations,  for  efficiently 
distributing/delivering  rations, 
and  for  automatically  monitoring 
and  managing  the  stored  rations. 


Methods  and  techniques  by  which 
principles  of  material  science, 
energy  utilization,  and  operations 
research  are  applied  in  developing 
food  service  equipment  that  is 
lightweight  and  durable,  that 
operates  with  minimum  energy  input, 
and  that  is  configured  for  optimum 
human  use. 


Techniques  by  which  principles  of 
chemical  extraction/ absorption, 
marine  biology,  plant  physiology, 
and  microbial  conversion  of  biomass 
are  applied  by  an  individual  to 
obtain  sustenance  from  the  land, 
air,  or  ocean. 


Technologies 

AI  Selection 
Handling 
Heating/Cooling 
Disposal 


Tailoring 

Distribution/ 

Delivery 

Monitoring/ 

Management 

Materials 

Engineering 

Energy 

Utilization 

Functional 

Optimization 

Biotechnology 
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6 .  Evaluating  the  Results  of  the  Exercise 


The  results  of  analyzing  the  entire  (DOD-wide)  sample  by  the 
method  explained  in  Section  II. 4  is  shown  in  Table  8: 

Table  8.  DOD-Wide  Priorities 

Priority  Priority 

Value  Benefit  Value  Benefit 

.327 

Strength  &  Alertness 

.02955 

(general)  Preparation 

.28273 

Satiety 

.02529 

Opening 

.07539 

Weight 

.0208 

Decontamination 

.06963 

Volume 

.01897 

Durability 

.05608 

Taste 

.01605 

Disposal 

.03535 

Heating 

.01141 

Recognizability 

.03171 

Variety 

Examples  of  partitioning  the  total  s 

ample  into  K— two,  and  three 

groups  are  shown  in  Appendix  B. 

Next,  Table  9  shows  the 

funding  and 

normalized  funding 

applicable  to  the  17  proposals, 

as  was  introduced  in  Section  II. 2: 

Table  9.  Normalization  of  Funding 

PROPOSAL 

%  FUNDING 

NUMBER 

FUNDING 

(FUNDING/TOTAL  FUNDS) 

FTB1259 

73 

4 

FTB1260 

210 

12 

FTB1262 

95 

5 

FTB1265 

185 

10 

FTB1266 

104 

6 

FTB1270 

125 

7 

FTB1271 

104 

6 

FTB1276 

104 

6 

FTB1279 

90 

5 

MSR1426 

144 

8 

MSR1449 

80 

5 

MSR1485 

35 

2 

MSR1486 

70 

4 

MSR1497 

52 

3 

MSR1498 

75 

4 

MSR1509 

95 

5 

MSR1511 

123 

7 _ 

TOTAL  FUNDS: 

:  1764 

AVERAGE:  5.8 
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Figure  3  recapitulates  the  remaining  steps  taken  to  arrive  at  a 
total  score  for  work  unit  FTB1259.  Note  that  although  it  is  more 
detailed  than  Figure  2,  the  same  basic  form  applies. 

Within  the  context  of  the  report,  space  limitations  became  a 
concern  regarding  the  depiction  of  the  FEA  process.  The  total  number  of 
branches  extending  from  a  work  unit  is  the  product  of  the  applicable 
technologies  times  the  benefits.  Thus,  up  to  390  branches  would  be 
required.  Obviously,  this  would  have  resulted  in  an  impossibly  complex 
diagram.  The  compromise  used  was  to  select  a  small  number  of 
technologies,  and  compute  findings  on  that  basis.  However,  this  action 
was  taken  for  reporting  purposes  only.  Since  electronic  data  processing 
techniques  allow  the  evaluation  of  large  numbers  of  technologies  and/or 
benefits,  no  such  compromise  is  necessary  during  actual  applications  of 
the  FEA  process.  Moreover,  it  is  again  emphasized  that  the  goal  of  the 
exercise,  rather  than  concentrating  on  perfection  of  examples,  was  to 
demonstrate  the  process,  itself,  in  a  start-to-f inish  manner. 

Proceeding  from  the  left,  Figure  3  follows  the  process  described 
in  Section  II.  The  initial  4%  of  total  requested  technology  base 
resources  comes  from  Table  9.  These  resources  are  then  distributed  to 
its  technologies  as  indicated  by  the  proportion  beneath  each  technology. 
Hence,  Dehydration's  share  of  FTB1259's  resources  is  4%  *  10%  or  .4%  of 
all  technology  base  resources.  This  is  the  estimated  cost  of  pursuing 
that  branch,  and  is  applied  to  all  benefits  anticipated  to  result  from 
this  work  unit- technology  combination.  The  next  column  identifies  the 
benefits  by  name.  Following  this,  the  median  probabilites  applicable  to 
each  work  unit-technology-benefit  branch  provided  by  the  technological 
evaluations  are  listed  under  'RAW'  BENEFIT.  The  'Raw'  Benefit/Cost 
column,  which  is  the  quotient  of  the  'RAW  Benfit'  column  divided  by  the 
main  branch's  resources,  is  included  to  allow  the  reader  a  preliminary 
look  at  these  relationships;  before  being  multiplied  by  the  DOD-Wide 
Priorities,  to  arrive  at  each  branch's  final  'Score'.  The  sum  of  these 
branch  scores  is  the  score  attained  by  work  unit  FTB1259,  and  is  listed 
at  the  lower -right  corner  of  Figure  3. 

Analogous  computations  determined  scores  for  the  remaining  16 
work  units.  The  results  are  listed  in  Table  10. 
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CD  H  TJ 


TECHNOLOGY 


I-> 
I-> 

Dehydration  I-> 

I-->  .4%  . I-> 

I  (10%  of  I-> 

I  FTB  1259)  I-> 

I  I-> 

I 
I 

I  I-> 

I  Nutritional  I-> 

I  Engineering  I-> 

I-->  .8%  . I-> 

I  (20%  of  I-> 

I  FTB  1259)  I-> 

I  I-> 

I 
I 

I  I-> 

1  I  Modularization  I-> 

2  -->  I-->  .4  % . I-> 

5  I  (10%  of  l-> 

9  I  FTB  1259)  I-> 

I  I-> 

(4%  I 

of  I 

Tech  I  Performance  I-> 

Base)  I  Enhancement  I-> 

I-->  .4  % . I-> 

I  (10%  of  I-> 

I  FTB  1259)  I-> 

I 
I 

I  I-> 

I  I-> 

I  Tailoring  I-> 

I-->  2  %  l-> 

(50%  of  I-> 

FTB  1259)  I-> 

I-> 


Benefit 

9 

RAW' 

Name 

Benefit 

Weight 

.446 

Volume 

.  446 

(general) 

Prep 

.396 

Opening 

.244 

Recognizability 

.208 

Taste 

.254 

Variety 

.236 

Weight 

.592 

Volume 

.592 

S  &  A 

.528 

Satiety 

.414 

Recognizability 

.236 

Taste 

.264 

Variety 

.264 

Weight 

.464 

Volume 

.  464 

(general) 

Prep 

.350 

Opening 

.336 

Recognizability 

.208 

Variety 

.236 

S  &  A 

.414 

Satiety 

.364 

Opening 

.336 

Taste 

.228 

Variety 

.239 

Weight 

.446 

Volume 

.446 

(general) 

Prep 

.396 

Opening 

.290 

Recognizability 

.186 

Taste 

.254 

Variety 

.546 

'RAW' 


Benefit/  DOD 

Branch 

Cost 

Priority 

Score 

111.50 

0.07539 

8.406 

111.50 

0.06963 

7.764 

99.00 

0.02955 

2.925 

61.00 

0.02529 

1.543 

52.00 

0.01141 

0.593 

63.50 

0.05608 

3.561 

59.00 

0.03171 

1.871 

Sub -Total : 

26.663 

74.00 

0.07539 

5.579 

74.00 

0.06963 

5.153 

66.00 

0.32700 

21.582 

51.75 

0.28273 

14.631 

29.50 

0.01141 

0.337 

33.00 

0.05608 

1.851 

33.00 

0.03171 

1.046 

Sub -Total : 

50.178 

116.00 

0.07539 

8.745 

116.00 

0.06963 

8.077 

87.50 

0.02955 

2.586 

84.00 

0.02529 

2.124 

52.00 

0.01141 

0.593 

59.00 

0.03171 

1.871 

Sub -Total : 

23.997 

103.50 

0.32700 

33.844 

91.00 

0.28273 

25.728 

84.00 

0.02529 

2.124 

57.00 

0.05608 

3.197 

59.75 

0.03171 

1.895 

Sub -Total : 

66.789 

22.30 

0.07539 

1.681 

22.30 

0.06963 

1.553 

19.80 

0.02955 

0.585 

14.50 

0.02529 

0.367 

9.30 

0.01141 

0.106 

12.70 

0.05608 

0.712 

27.30 

0.03171 

0.866 

Sub -Total : 

5.870 

Total : 

173.496 

Figure  3.  Scoring  Proposal  FTB  1259 
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Table  10.  Prioritization  of  the  Candidate  Work  Units 


Rank 

Proposal 

Number 

Score 

1 

MSR1449 

486.463 

2 

MSR1485 

181.284 

3 

FTB1259 

173.496 

4 

MSR1497 

136.069 

5 

MSR1498 

129.805 

6 

MSR1486 

79.263 

7 

MSR1426 

71.034 

8 

FTB1265 

63.067 

9 

FTB1276 

60.643 

10 

FTB1266 

53.062 

11 

MSR1511 

44.674 

12 

MSR1509 

42.987 

13 

FTB1279 

33.666 

14 

FTB1260 

25.283 

15 

FTB1270 

16.718 

16 

FTB1262 

14.668 

17 

FTB1271 

7.260 

7 .  Conclusions 

Almost  unilaterally,  MSR's  scored  higher  than  technology  base 
proposals.  Of  course,  the  inclusion  of  these  types  of  proposals  was  not 
an  objective  of  this  FEA.  However,  it  must  be  concluded  that  the  process 
developed  in  this  FEA,  in  and  of  itself,  might  not  lead  to  an  acceptably 
'balanced'  DOD  Food  Program. 

Why  did  the  MSR's  dominate  the  prioritization?  One  answer  may  lie  in 
the  users'  priorities,  which  have  been  shown  to  impact  heavily  upon  a 
proposal's  final  score.  Possibly  these  representatives  are  still  mired  in 
near-term  thinking.  However,  another  plausible  hypothesis  is  that  they 
are  thinking  long  terra,  but  do  not  anticipate  substantial  departures  from 
their  current  priorities  in  the  future.  In  either  case,  these  priorities 
are  more  compatible  with  the  near-term  focus  common  to  most  MSR's.  Even 
the  single  exception  in  the  top  seven  ranks  seems  to  confirm  this 
conclusion,  for  the  objectives  and  approach  of  FTB1259  (Universal  Nutrient 
Dense  Components)  make  it  extremely  compatible  with  some  of  the  most 
highly-prioritized  benefits  in  Table  8.  Alternatively,  two  MSR's  did 
score  relatively  low.  While  the  technologies  pursued  in  MSR1509  may  have 
some  peripheral  relevance  to  individual  rations,  that  was  certainly  not 
the  primary  focus  of  that  effort.  Additionally,  the  objectives/approach 
to  this  Air  Force  project  may  have  been  more  difficult  to  generalize.  The 
reasons  for  MSR1511's  relatively  low  rank  are  less  obvious,  for  its  focus 
on  ease  of  heating  is  certainly  of  some  importance  to  the  users.  The 
explanation  is  twofold.  Firstly,  referencing  the  average  %  funding 
computed  in  Table  9,  MSR1511  was  a  relatively  high  consumer  of  resources. 
Thus,  in  order  to  maintain  a  competitive  score,  it  would  have  to  have  been 
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rated  relatively  highly  by  both  technological  evaluators  and  users. 
More  importantly,  while  ease  of  heating  is  undeniably  an  important 
benefit,  it  is  still  only  one  benefit.  Thus,  it  is  almost  assured  a 
relatively  low  score,  when  assessed  in  relation  to  other  proposals 
which  pursue  multiple  benefits.  Another  important  conclusion  to  be 
drawn  from  this  is  that  the  FEA  process  will  work  best  when  proposals 
can  be  anticipated  to  provide  nearly  uniform  numbers  of  benefits. 

Another  important  issue  is  raised  by  the  last -place  positioning 
of  FTB1271.  'Attainment  and  Validation  of  Microbial  Control'  can  be 
anticipated  to  make  important  advancements  regarding  shelf  life. 
However,  users  may  be  more  inclined  to  perceive  shelf  life  as  a 
prerequisite,  rather  than  a  benefit.  Hence,  one  limitation  of  the 
FEA  process  concerns  the  distinction  between  benefits  and  other 
important  areas  of  research  in  which  the  linkage  to  benefits  may  not 
be  apparent.  However,  when  the  FEA  process  does  prioritize  a 
proposal  as  low,  the  process  has  merit  even  if  that  rank  is 
eventually  rejected.  The  process,  itself,  has  focused  thinking;  as 
to  why  the  proposal  attained  a  low  score,  and  why  it  may  still 
constitute  a  viable  avenue  for  technology  base  research.  Since  it  is 
possible  that  actual  reviews  of  the  FOOD  Program  may  ask  similar 
questions,  this  type  of  focus  allows  Natick  to  consider  its  position 
on  a  questionable  proposal  in  advance;  to  be  'proactive',  rather  than 
reactive . 

Finally,  since  a  review  of  Table  9  indicates  that  FTB1259  was  an 
above-average  consumer  of  resources,  how  did  it  attain  second  place 
among  technological  proposals?  In  part,  this  is  due  to  the 
relatively  high  values  the  users  placed  upon  behaviorally-oriented 
benefits.  However,  it  is  also  due  to  high  estimated  likelihood  of 
success  regarding  these  benefits  by  the  technological  evaluators. 
While  there  is  nothing  inherently  wrong  with  this,  it  does  lead  to 
questions  of  comparability  regarding  the  inherent  difficulty  of 
achieving  the  objectives  of  competing  proposals.  That  is,  the 
potential  exists  for  some  proposals  to  score  very  well  during 
technical  evaluations  merely  because  their  objectives  are  not  very 
'challenging'.  Even  when  proposals  are  not  very  challenging,  they 
may  never- the- less  be  worth  pursuing.  The  question  is  not  their 
liability .  Rather,  the  real  issue  is  whether  or  not  such  proposals 
belong  in  the  technology  base. 

It  is  recommended  that  the  methodology  developed  in  this  FEA  be 
incorporated  into  the  DOD  Food  Program's  process  for  prioritizing 
technology  base  proposals.  This  recommendation  is  based  upon  the 
achievement  of  the  FEA's  objectives.  A  methodology  was  developed  by 
which  input  from  large  numbers  of  military  personnel,  used  in 
conjunction  with  technological  evaluations  of  proposed  research  and 
related  cost  data,  can  produce  useful  information.  However,  such 
information  must  be  considered  within  the  larger  context  of 
managerial  judgement  and  common  sense. 
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Appendix  A 

Organizations  Representing  the  User  Community 


Army  Organizations 


US  Army  Air  Defense  School 
Ft  Bliss,  TX 

US  Army  Armor  School 
Ft  Knox,  KY 

US  Army  Logistics  Center 
Ft  Lee,  VA 

US  Array  Aviation  Logistics  School 
Ft  Eustis,  VA 

US  Army  Military  Police  School 
Ft  McClellan,  AL 

US  Army  Combined  Arms  Center 
Ft  Leavenworth,  KS 

US  Army  Engineer  School 
Ft  Belvoir ,  VA 

US  Army  Intelligence  School 
Ft  Huachuca,  AZ 

US  Army  Ordnance  Missile 
&  Munitions  School 
Redstone  Arsenal,  AL 


US  Army  Aviation  Center 
Ft  Rucker,  AL 

US  Army  Infantry  School 
Ft  Benning,  GA 

US  Army  Ordnance  School 
Aberdeen  Proving  Grounds,  MD 

US  Army  Quartermaster  School 
Ft  Lee,  VA 

US  Army  Transportation  School 
Ft  Eustis,  VA 

US  Army  Communications  School 
Ft  Gordon,  GA 

US  Army  Field  Artillery  School 
Ft  Sill,  OK 

US  Army  Chemical  School 
Ft  McClellan,  AL 

US  Array  John  F.  Kennedy 
Special  Warfare  Center 
Ft  Bragg,  NC 


Navy  Organizations 


US  Naval  Construction  Battalion 
Norfolk,  VA 

US  Naval  Air  Systems  Command 
Washington,  DC 


US  Naval  Special  Warfare  Group  1 
Coronado,  CA 

US  Naval  Development  Center 
Warminster,  PA 
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Appendix  A  (Cont'd.) 

Organizations  Representing  the  User  Community 


Marine  Corps  Organizations 

MC  Combat  Development  Command  MC  Research,  Development  & 

Warfighting  Center  Acquisition  Command 

Quantico,  VA  Rosslyn,  VA 

Headquarters  Marine  Corps 
Washington,  DC 


Air  Force  Organizations 


Alaskan  Air  Command 
Elmendorf  AFB  AK 


Strategic  Air  Command 
Offutt  AFB  HI 


AF  Logistics  Command 
Wright-Patterson  AFB  OH 


AF  Space  Command 
Peterson  AFB  CO 


AF  Systems  Command 
Andrews  AFB  MD 

Air  Training  Command 
Randolph  AFB  TX 

Air  University 
Maxwell  AFB  AL 


Tactical  Air  Command 
Langley  AFB  VA 

USAF  Academy  Preparatory  School 
Colorado  Springs  CO 

Electronic  Security  Command 
San  Antonio  TX 


Military  Airlift  Command 
Scott  AFB  IL 


HQ  Air  Force  Reserve 
Robins  AFB  GA 


HQ  Pacific  Air  Forces 
Hickam  AFB  HI 


AF  Commissary  Service 
Kelly  AFB  TX 


HQ  US  Air  Forces  in  Europe 
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Appendix  B.  Examples  of  Clustering 


K-2  Groups 

Group  1  Group  2 

(70%)  (30%) 


Priority 

Value 

Benefit 

Priority 

Value 

Benefit 

.279 

S  &  A 

.413 

S  &  A 

.269 

SATIETY 

.185 

SATIETY 

.  101 

TASTE 

.176 

WEIGHT 

.062 

VOLUME 

.17 

VOLUME 

.06 

VARIETY 

.022 

HEATING 

.057 

WEIGHT 

.021 

PREPARATION 

.039 

HEATING 

.012 

OPENING 

.028 

PREPARATION 

* 

DURABILITY 

.024 

RECOGNIZABILITY 

* 

DECONTAMINATION 

.023 

OPENING 

* 

DISPOSAL 

.021 

DECONTAMINATION 

* 

RECOGNIZABILITY 

.021 

DURABILITY 

* 

TASTE 

.016 

DISPOSAL 

* 

VARIETY 

Group  1 
(41%) 

Priority 

K-3  Groups 

Group  2 
(27%) 

Priority 

Group  3 
(31%) 

Priority 

Value 

Benefit 

Value 

Benefit 

Value 

Benefit 

.558 

SATIETY 

.321 

WEIGHT 

.230 

SATIETY 

.109 

VOLUME 

.299 

VOLUME 

.177 

S  &  A 

.092 

WEIGHT 

.294 

SATIETY 

.094 

TASTE 

.068 

HEATING 

.035 

HEATING 

.086 

WEIGHT 

.056 

PREPARATION 

.033 

PREPARATION 

.086 

VOLUME 

.036 

DECONTAMINATION 

.018 

OPENING 

.071 

VARIETY 

.033 

OPENING 

* 

S  &  A 

.058 

HEATING 

.031 

DURABILITY 

* 

DURABILITY 

.039 

OPENING 

.017 

DISPOSAL 

* 

DECONTAMINATION 

.038 

PREPARATION 

* 

S  &  A 

* 

DISPOSAL 

.031 

DURABILITY 

* 

RECOGNIZABILITY 

* 

RECOGNIZABILITY 

.031 

DECONTAMINATION 

* 

TASTE 

* 

TASTE 

.03 

DISPOSAL 

* 

VARIETY 

* 

VARIETY 

.028 

RECOGNIZABILITY 

*  Group  was  inconsistent  on  the  indicated  benefits.  Inconsistency,  which  is 
discussed  in  considerable  detail  in  the  Saaty  text,  basically  refers  to  the 
degree  of  discontinuity  between  expressed  relationships,  and  those 
algebraically  implied  from  other  responses.  For  example,  if  x/y-1/2  and 
y/z-1/4,  then  the  implied  relationship  x/z  is  1/8.  If  an  expressed 
comparison  of  x  to  z  does  not  yield  1/8,  the  degree  of  difference  is  an 
indication  of  inconsistency.  In  the  FEA  exercise,  when  the  set  of  data  taken 
as  a  whole  was  not  sufficiently  consistent,  all  judgements  which  involved  a 
particular  benefit  were  removed  from  the  set;  and  the  consistency  of  the 
residual  set  recalculated.  If  no  single  source  of  inconsistency  could  be 
identified,  the  process  was  repeated,  removing  pairs  of  benefits,  until 
satisfactory  consistency  in  the  residual  set  was  achieved. 
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